Correction of Faded Colors in an Image Using an Integrated Multi-Scale GrayWorld Algorithm

The correction of faded colors in old pictures, prints, and paintings is an interesting issue for color image processing. Several techniques have already been introduced to enhance faded color images, many of which approach the problem as color cast removal and use global illuminant estimation methods, such as the gray world or Von Kries assumptions. However, the use of simple global operators to eliminate the illuminant effects is not always suitable for enhancing faded images. Therefore, this article presents a color correction algorithm based on a multi-scale gray world algorithm for faded color images. First, the proposed method adopts a local process using multi-scale filters, and coefficients are obtained for each filtered image. Integration of the coefficients using weights is then performed to calculate the correction ratio for the red and blue channels in the gray world assumption. Finally, the corrected image is obtained by applying the integrated coefficients to the gray world algorithm. In experiments, the proposed method is able to reproduce corrected colors for both wholly and partially faded images, in contrast to previous methods. The proposed method also enhances the visibility of the input images using multi-scale processing. c © 2013 Society for Imaging Science and Technology. [DOI: 10.2352/J.ImagingSci.Technol.2013.57.6.060552] INTRODUCTION With the increasing interest in digital color correction of old pictures, prints, and paintings, special digital image processing techniques have recently been developed for the specific purpose of image enhancement and restoration. Old-style media invariably lose their original color over time. Thus, the phenomenon of color fading and correction of faded images have been explored through various experiments, in an attempt to correct faded images. For example, Frey1 and Wilhelm2 investigated the effect of temperature, humidity, and illuminants on color fading, and found that the fading rate differed according to the ink property, temperature, humidity, and illuminant. As IS&T Members. Received Mar. 15, 2012; accepted for publication Dec. 16, 2013; published online Mar. 12, 2014. Associate Editor: Jon Yngve Hardeberg. 1062-3701/2014/57(6)/060505/14/$20.00 such, color photoprints or documents fade according to the illuminant, color, and area in the image. However, old-style media will be distorted over time by the storage environment (illuminants, temperature, humidity, dye characteristics, and so on). Because of the very irregular nature of such transformations, we cannot properly estimate the transformations’ characteristics. Therefore, we instead focus on finding a suitable compensation and enhancement method for digitally acquired faded images. If the phenomenon of color fading is loosely regarded as a problem of color cast, computational color constancy methods might be applied to enhance the color of faded images. The general computational constancy model assumes that the measurement made by a sensor is proportional to the product of the reflectance and the illuminant.3 The reflectance of an object can be calculated from the color by estimating the illuminant. Two basic but widely used computational constancy methods are the global gray world and white patch assumptions (the latter also known as Von Kries),3 where the gray world assumption estimates the illuminant using the average color of the pixels, while the white patch method assumes that the color ‘‘patch’’ in the scene that reflects the maximum light possible for each band should be regarded as the reference white. However, the global white patch assumption does not hold for the correction of faded images, due to the change of the dye properties by illuminants. In other words, a white area in a printed color photo or document has no dyes, which usually means that it is subject tominimal changes compared to areas covered with dye. A combination of these two methods has also been proposed, which uses a quadratic correction of two channels in order to satisfy both the gray world and white patch assumptions.4 Yet, all these methods are based on the global adaptation mechanism of human vision and work point-wise on image data. Thus, while simple and efficient, such methods cannot correct partially faded images. J. Imaging Sci. Technol. 060505-1 Nov.-Dec. 2013 Kyung et al.: Correction of faded colors in an image using an integrated multi-scale gray world algorithm Land and McCann5 later formulated a model of color constancy in human vision. This model, known as Retinex, can be roughly but simply described as a local white patch approach. The Retinex model of human vision has been the subject of study through the years and it has led to a number of different implementations and derived methods. Some recent instances of work derived from the Retinex theory can be found in Random Spray Retinex (RSR),7 an implementation that uses stochastic sampling, and Spatio-Temporal Retinex-Inspired Envelope with Stochastic Sampling (STRESS),8 the latter actually being a more complex framework for computing spatio-temporal image envelopes via stochastic sampling. Related to but not directly deriving fromRetinex are also Automatic Color Equalization (ACE),9,10 an automatic local gray world approach, and RACE,11 a framework based on stochastic sampling for merging local white patch and gray world approaches. During the later stages of his research on the human visual system, Land came to formulate a different model which he named ‘‘designator’’ (in fact he even gave two distinct formulations).12 From Land’s designator was born one of the most famous algorithms related to Retinex, the so-called NASA Retinex13,14 (although in this particular case the term Retinex is a kind of misnomer). Various methods deriving from theNASAapproach have beenwidely used, due to their accurate local illuminant estimation and visibility improvement, even given the presence of artifacts like halo effects. Jobson et al developed the designator theory into the single-scale Retinex (SSR) method and multi-scale Retinex (MSR)method as a combined formof the SSR method.13,14 As the MSR method initially experienced problems related to appropriate values for the parameters, chromatic imbalance, color distortion, noise, and graying out, a lot of research has been dedicated to improving these issues. Thus, a multi-scale Retinex with color correction was developed to overcome the graying-out phenomenon in large uniform areas in an image by adopting a color correction function to control the saturation of the final rendition.15 In a recent article, an integrated multi-scale Retinex (IMSR) algorithm was introduced to improve the visibility in dark shadow areas of natural color images, while preserving a pleasing contrast without banding artifacts.16 In this case, a Gaussian pyramid decomposition is used to reduce the computational time for generating a large-scale surround, while an integrated surround value for the luminance is applied to each channel to preserve the color balance in RGB color space. However, IMSR cannot correct faded images due to its preservation of the color balance in the input image. Accordingly, this article proposes a multi-scale gray world algorithm to correct faded color images. First, multiscale masks, such as average filters, are used to estimate the faded color. Next, to apply the gray world algorithm, coefficients are calculated for each pixel in each filtered image, where the coefficients represent the correction ratios between green and the other color channels in the gray world assumption theory. In addition, an integrated coefficientmap is also generated using the weighted sum of the coefficients calculated for each scale. The corrected image is then obtained by applying the integrated coefficient map to the gray world algorithm. Furthermore, the proposed method enhances the visibility of the input images using an additional tone reproduction process. First, the average filtered images are converted to luminance images to obtain an enhanced luminance image based on the weighted sum of the blurred luminance images. The enhanced luminance image is then used to calculate coefficients. Experiments show that the proposed method is able to enhance the color in the cases of both wholly and partially faded images. PREVIOUSWORKS Most color constancy methods assume that the perceptual color is a product of the reflectance and the illuminant. The reflectance of an object can be calculated from the perceptual color by estimating the illuminant. These methods are based on the theory of color image formation. The intensity I measured by a camera sensor at position (x, y) can be modeled as I(x, y)= E(x, y) ∫ R(λ, x, y)L(λ)S(λ)dλ, (1) where E(x, y) is the scaling factor resulting from the geometry of the patch at position (x, y), R(λ, x, y) denotes the reflectance at position (x, y),L(λ) is the radiance given off by the light source, and S(λ) describes the sensitivity of the sensors.17 It is assumed that the response functions of the sensors have a very narrow band, i.e., they can be approximated by a delta function. Let λi with i ∈ {r, g, b} be the wavelengths to which the sensors respond. Under a nonuniform illuminant, the intensity measured by the sensor can be modeled as follows3: Ii(x, y)= E(x, y)Ri(x, y)Li(x, y), (2) where E(x, y) is the factor that depends on the scene geometry, Ri(x, y) is the reflectance for wavelength λi, and Li(x, y) is the irradiance at position (x, y) for wavelength λi. Gray World Assumption Algorithm The gray world assumption was originally proposed by Buchsbaum,18 and it estimates the illuminant by assuming the existence of a certain standard spatial spectral average for the total visual field. For image I(x, y) with the size M×N, IR(x, y), IG(x, y), and IB(x, y) denote the red, green, and blue channels of the image, respectively. The average values, Ravg, Gavg, and Bavg, are then calculated as